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Abstract: Automation and advanced technologies have increased the need for a better 
understanding of the skills necessary to have a globally competitive workforce. This study used 
data from the Program for the International Assessment of Adult Competencies to compare 
problem solving skills in technology rich environments among adults in South Korea, Germany, 
Singapore, Japan, Canada, Estonia, the United Kingdom, the United States, and Australia. 
Overall, the United States had the lowest scores among all countries, and in all countries scores 
declined with age. The United States had higher proportions of survey participants in the lowest 
skill category and lower proportions in the top skill categories. The results of this study suggest 
changes in the United States educational and lifelong learning systems and policies may be 


necessary to ensure all adults have the necessary skills in a competitive workforce. 
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Problem-Solving Skills of the U.S. Workforce and Preparedness for Job Automation 

Introduction 

In recent years, overall labor force participation rates (LFPR) have been declining in the 
United States. While population aging has been a contributing factor, declines in LFPR among 
working-age adults, particularly males, has also played a role. For example, between 1997 and 
2017 the LFPR for males in the 25 — 34 age group declined from 93% to 89% and for the 45 - 54 
age group it declined from 90% to 86%. LFPR for females in those age groups also declined, but 
not to the same extent as males (OECD Statistics, 2018). There are multiple reasons for this 
decline, but the increased presence of job automation (e.g., computer programs, robots) in a 
variety of industries is an important factor for reduced LFPR (Abraham & Kearney, 2018). 
Advances in technology and automation are expected to suppress the number of jobs available 
(Arntz, Gregory, & Zierahn, 2016; Frey & Osborne, 2017), and low-skilled (e.g., manual labor) 
workers are more likely to be impacted than high-skilled workers (Arntz et al., 2016; Frey & 
Osborne, 2017; White House, 2016). Frey and Osborne (2017) estimate that about 47% of jobs in 
the United States are at risk of being replaced by automation over the next two decades, but there 
are substantial disparities depending on wages, qualifications and demographic background. For 
instance, over 80% of jobs paying less than $20 per hour could be automated, whereas only 4% 
of jobs paying $40 or more could be automated (White House, 2016). Consequently, workers 
with lower levels of education are more vulnerable to the effects of job automation (Frey & 
Osborne, 2017). In the United States, Blacks and Latinos have lower levels of education and are 
more likely to work in low-skilled occupations when compared to Whites (Gabe, Abel, & 
Florida, 2018; Nettles, 2017), making some racial/ethnic minorities especially vulnerable to job 


loss due to automation. It should be noted that technological advances may exacerbate income 


inequality as higher skilled workers are more likely to benefit (Hoeller, Joumard, & Koske, 
2014). 

The Program for the International Assessment of Adult Competencies (PIAAC) is an 
international skills assessment survey developed by the Organisation for Economic Cooperation 
and Develoment (OECD) and implemented by member countries. PIAAC measured literacy, 
numeracy and problem solving skills in technology rich environments (PSTRE). The PIAAC 
PSTRE measure was designed to quantify a set of abilities to use technology (e.g., computer 
literacy) and higher cognitive skills of adults (Reder, 2015), which is especially important to 
examine given the increased need to advanced technical skills. 

The Economist Intelligence Unit (EIU, 2018) created an index to identify countries most 
prepared for the coming wave of employment automation. Preparedness is evaluated based on 
the relevant policies and infrastructure in place in terms of technological innovation, education 
and the labor market. One may say that a high index ranking indicates the potential for greater 
technological advancement, workforce competencies, and employment security. Twenty-five 
countries were evaluated to identify government efforts to prepare for technological advances in 
three main policy areas: innovation environment, education policies, and labor market policies. 
Based on these criteria, 10 countries with the highest overall indices (i.e., most prepared for the 
job automation) include in this rank order: South Korea, Germany, Singapore, Japan, Canada, 
Estonia, France, the United Kingdom, the United States, and Australia (The Economist 
Intelligence Unit, 2018). 

The purpose of the current study is to examine the distributions of the job automation 
readiness index created by the EIU (2018) and compare to the data to PIAAC’s PSTRE results. 


(PSTRE is more fully described in the methods section). To assess preparedness for the 


anticipated further technological advancement and job automation, it is necessary to analyze 
problem-solving skills in the uses of digital technologies to inform future training and 
educational programs. The PSTRE skills assessment is an indispensable step to document the 
current skill distributions and help develop strategies to improve PSTRE of the vulnerable (e.g., 
low qualification) sub-populations (Rouet et al., 2009). 
Literature Review 

Advances in technology, such as an increase in the use of robotics in the workplace, 
require ongoing investments in technology-related training programs for workers. Particular job 
skills (e.g., simple manual labor) are increasingly becoming obsolete as a result of technological 
advances (Baptista, 2016; Guzman, Pawliczko, Beales, Till, & Voelcker, 2012). Familiarity with 
technology is thus related to workforce participation and income. For instance, the OECD reports 
that technological proficiency and use of technology is associated with an increase in labor force 
participation and higher wages (OECD, 2015). These trends suggest that there is a growing need 
for promoting human capital in general and technology-related occupational skills in particular 
through lifelong training for workers of all ages (Baptista, 2016; Guzman et al., 2012). While 
many sectors have undergone rapid advances in digital technologies, others, such as health care, 
government, and personal services have not advanced as rapidly. This may change in the future 
(Berger, 2016). Technology advances have many benefits but can result in substantial changes 
for workers. Automation changes the way work is conducted by eliminating some jobs while at 
the same time creating new jobs (National Academies of Sciences Engineering and Medicine, 
2017). Preparing the workforce for these changes is critical to create a viable workforce and to 


remain economically competitive. 


In addition to ongoing investments in human capital, the education systems and policies 
in the United States need to enable workers to navigate increasingly automated workforces. For 
example, early interventions through formal education focusing on science, technology, 
engineering, and mathematics (STEM) are beneficial to ensure future workers are prepared for a 
workplace dominated by machines (Kim, Kim, & Lee, 2017). There is also a need for the 
implementation of lifelong learning opportunities in order to maintain and improve technology- 
related skills even beyond formal education. South Korea, Germany, Singapore, and Japan, the 
mostly highly rated countries for automation readiness, have already undertaken initiatives for 
curriculum reform, lifelong learning, and occupational training and their governments have made 
substantial investments in research and development in support of developing new technologies 
(EIU, 2018). An emphasis on vocational programs in Singapore and South Korea contribute to 
these countries’ high ranking on the automation readiness index (EIU, 2018). Similarly, 
Germany’s high ranking in the automation readiness index can be illustrated by the country’s 
strong system of vocational education and apprenticeship programs (Spitz-Oener, 2006; EIU, 
2018). Finally, similar initiatives towards automation are seen in Japan, as the country’s 
government invests millions of dollars’ worth of funds to research organizations developing new 
technologies in artificial intelligence (AI) and robotics (EIU, 2018). 

As noted previously, low-skilled workers are especially vulnerable to job automation. To 
make the situation worse, growing minimum wages have increased the likelihood that employers 
invest in technology to automate jobs previously held by low-skilled workers, those who are 
perhaps the most in need for training to improve skills (Lordan & Neumark, 2017). Furthermore, 
low-skilled workers have fewer education opportunities. Indeed, participation in job-related 


training such as on-the-job training is highest among high-skilled workers who are at the lowest 


risk for being replaced by automation technology (Nedelkoska & Quintini, 2018). Opportunities 
for participation in educational activities over the life course are necessary to have a workforce 
that is well-prepared for even more advanced technologies. Taken together, an examination of 
the skill distributions among current U.S. workforce in the global context is an indispensable first 
step to inform education systems and policies in the era of job automation. 
Research Questions 
To better understand the preparedness of workers in the United States to function in a 
technologically advanced economy, this study explores the PSTRE in the PIAAC data and 
compares it to the ten most job automation-ready countries (listed in the next sections). 
Specifically, we addressed the following research questions: 
1. Compared to the countries that are most prepared for job automation, what are PSTRE 
skills in the United States? 
2. Compared to the countries that are most prepared for job automation, what proportions of 
U.S. adults have insufficient proficiency in PSTRE? 
3. For the United States, is there any differences in the PSTRE proficiencies across 
demographic characteristics including age groups, and race/ethnic groups? 
4. Compared to the countries that are most prepared for job automation, what proportion of 
U.S. adults use complex computer skills at work? 
Data Sources and Methods 
We used 2012/2014 PIAAC data to compare PSTRE scores among South Korea, 
Germany, Singapore, Japan, Canada, Estonia, the United Kingdom, United States, and Australia 
(OECD, 2014). France was excluded from analysis due to the non-participation in the PSTRE 


assessment in PLAAC. PIAAC was implemented by 38 countries (National Center for Education 


Statistics, 2018) and all but five of the countries implemented the PSTRE assessment (OECD, 
2018). PIAAC’s goal is to assess and compare basic skills and a broad range of competencies of 
adults, including literacy, numeracy, and PSTRE as well as an extensive array of 
sociodemographic characteristics (Goodman, Finnegan, Mohadjer, Krenzke, & Hogan, 2013). In 
the context of the PIAAC survey, PSTRE is defined as: 

Problem solving in technology-rich environments involves using digital 

technology, communication tools and networks to acquire and evaluate 

information, communicate with others and perform practical tasks. The first 

PIAAC problem solving survey will focus on the abilities to solve problems for 

personal, work and civic purposes by setting up appropriate goals and plans, 

accessing and making use of information through computers and computer 

networks (Rouet et al., 2009, p. 9). 

PIAAC data provide survey weights to adjust for the complex sampling design for 
estimating nationally representative figures (OECD, 2016b). PIAAC provides a set of 10 
plausible values or statistically estimated PSTRE scores (range: 0-500) based on the respondents’ 
performance on the PSTRE-related tasks (Kirsch & Thorn, 2013). To facilitate meaningful 
interpretation of results, proficiency levels and specific score cut-points have been suggested by 
PIAAC. PSTRE scores were classified into four levels, below level 1 and levels 1 through 3. 
Those who score below level | are unable to use widely available and familiar technology 
applications, such as email or a web browser or complete simple forms of reasoning, whereas 
those at level 1 are able to use generic applications where little or no navigation is required. 
Levels 2 and 3 utilize increasingly complex technology skills, such as generic and novel 


applications involving multiple steps and operators (OECD, 2016b). To document the use of 


computer skills at work, we examined the survey item --- “what level of computer use is needed 
to perform your job.” For purposes of this question, straightforward includes data entry and 
sending emails, moderate includes word-processing, spreadsheets, and database management, 
and complex computer use includes developing software or modifying computer games, 
programming using languages like java, SQL, PHP or Perl, or maintaining a computer network 
(OECD, 2010). 

We used the International Data Explorer (IDE) available through the National Center for 
Education Statistics (National Center for Educational Statistics, 2018) and the Organisation for 
Economic Cooperation and Development (OECD, 2016a). The IDE is an interactive online 
application to generate the inter/nationally representative descriptive summaries of PIAAC data. 
The IDE takes complex survey weights and plausible values into account to calculate summary 
statistics and to conduct simple statistical significance test. 

Results and Discussion 
Research Question 1: Compared to the countries that are most prepared for job automation, 
what are PSTRE skills in the United States ? 

Overall, the U.S. PSTRE score of 274 was significantly lower than other countries 
included in the analysis, with Japan having the highest overall score (294) followed by Australia 
(289), Singapore (287), Germany and South Korea (both 283), Canada (282), the United 
Kingdom (280) and Estonia (278) (see Table 1). 

[Insert Table 1 about here] 
Research Question 2: Compared to the countries that are most prepared for job automation what 


proportions of U.S. adults have insufficient proficiency in PSTRE? 
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The United States had the greatest proportion (23%) scoring below Level 1. Australia and 
Japan had the lowest proportions scoring below level one (12%) followed by South Korea (14%; 
see Figure 1). Only 36% of U.S. survey participants scored in the top two levels as compared to 
56% in Japan and 49% in Singapore. With 7% in the top Level (3), the United States did not 
have the lowest proportion, but did have the lowest proportion in the combined top two levels 
and the highest proportion in the bottom two levels. 

[Insert Figure 1 about here] 

Research Question 3: For the United States, are there any differences in PSTRE proficiencies 
across demographic characteristics including age groups, and race/ethnic groups? 

Patterns were similar for the 25 — 34 age group with U.S. score significantly lower than 
the other countries (see Table 1). Japan had the highest score (302) while the United States had a 
score of 283. For the 35 — 44 age group, Germany, Singapore, Japan, Canada, the United 
Kingdom, and Australia all scored significantly higher than the United States and the United 
States tied with Estonia for the lowest score for this age group. Patterns were not as consistent 
for the oldest two age groups. For the 45 to 54 age group, South Korea (261) and Estonia scored 
lower than the United States while Australia has the highest score for this age group (270). 
Australia had the highest average score (270) for the 55 to 65 age group followed by the United 
Kingdom (263), Japan (262), Canada, and the United States (both 261). It is interesting to note 
that as age increased, scores decreased in all countries. Also, Singapore PSTRE scores had the 
largest gap between the 25 — 34 and 55 — 65 age groups (54) while the United States had the 
smallest gap (22). 

Latinos and Blacks had far greater proportions scoring below Level 1 as compared to 


Whites (37%, 42%, and 16% respectively). In addition, lower proportions of Latinos and Blacks 
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attained scores in Levels 2 and 3 as compared to Whites (24%, 14%, and 43% respectively; see 
Figure 3). 

[Insert Figure 2 about here] 
Research Question 4: Compared to the countries that are most prepared for job automation, 
what proportion of U.S. adults use complex computer skills at work? 

In the United Kingdom, the United States, and Australia, 9% of respondents reported 
using complex skills at work while other countries reported between 6% and 8%. The United 
States had the highest proportion using only straightforward skills at work (37%) and also had 
the lowest proportion in the combined moderate and complex use of skills at work (63%). 
Interestingly, Estonia had the highest proportion (79%) in the combined moderate and complex 
categories (see Figure 3). 

The overall U.S. PSTRE scores were lower than the highest-ranking countries in the EIU 
index and were lower for most age groups. Of perhaps greater concern is the high proportion of 
U.S. survey participants who scored below level 1 along with the low proportion in the United 
States scoring in the top two levels (i.e. levels 2 and 3). Variations among age groups in PSTRE 
scores emphasize the need for ongoing skill upgrades to ensure continued employment and to 
have a workforce that can be competitive in a global economy. Variations within the United 
States based on race/ethnicity emphasize the critical need for improved educational opportunities 
for disadvantaged groups to reduce inequality. 

Implications for Policy and Practice and Future Research 

Gaining a better understanding of PSTRE skills distribution in the United States is 

important to ensure the economic security of individuals and to create a workforce that can be 


globally competitive, given the increasing reliance on technology within the workforce. Given 
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the PSTRE scores of U.S. adults, evaluation of how students might be educated differently to 
improve practical and/or job-related problem-solving skills needs to be a priority. 
Apprenticeships, which are very common in Germany, are one potential mechanism for 
improving technology skills. Community colleges have begun to expand their offering of 
apprenticeship programs, but because of funding models, these programs have been slow to 
develop (Holzer, 2017). Additional sources of funding are necessary to develop programs that 
will provide opportunities to improve the technical skills of adults in the United States. 
Employers have worked with community colleges to develop programs to meet their specific 
needs. For example, in order to build a technically skilled workforce, IBM partnered with ten 
community colleges to develop apprenticeship programs (Jordan, 2018). Development of 
educational practices to improve the technical skills of adults in the United States along with 
implementation of policies to fund the programs are necessary to remain competitive in a 
technology advanced society. It is also important that faculty be provided with resources and 
professional development opportunities so they are prepared to convey to students knowledge 
about advanced technologies. 

In addition to education for students in formal education institutions, lifelong educational 
opportunities for adults of all ages need to be evaluated to enable individuals to improve their 
skills. Without action to enhance access to educational opportunities, the U.S. workforce may 
continue to fall behind other OECD countries (OECD, 2013). Indeed, the OECD (2013) 
recognized in its policy recommendations the important role community colleges will play in 
improving the skills of U.S. adults: “Recommendation 3: Ensure effective and accessible 
education opportunities for young adults, using the strengths of the community college system to 


support and develop basic skills and offer substantive career options” (p. 45). Strong 
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relationships between employers and community colleges will be important to ensure vocational 
programs are available to provide students with the skills required for jobs in demand. 

Vanek (2017) suggested the following steps to improve PSTRE skills: 1) goal setting in 
order to improve task completion; 2) planning, self-organizing, which involved moving through 
a series of steps that require reflection and actions; 3) acquiring and evaluation of information; 4) 
monitoring progress by continuous evaluation of actions taken; and 5) making use of information 
(p. 10). Critical thinking skills, which include the ability to reason effectively, make judgements, 
and solve problems are an important component of PSTRE (Vanek, 2017) and are skills valued 
by employers. Incorporating these five steps into learning activities and course work may require 
professional development programs for faculty. For example, some faculty might not be well 
equipped to implement technology- based learning activities in the classroom, and may benefit 
from instruction to improve their skills (Adams, 2002). Professional instruction for faculty has 
also been shown to improve self-efficacy with technology as well as the intention to integrate 
technology into the classroom (Banas & York, 2014). In order to facilitate the improvement of 
PSTRE skills by disadvantaged groups, tutoring and other student supports may be necessary. 

The PIAAC PSTRE results are not entirely consistent with the EIU index, presumably 
because PSTRE is only a part of job automation preparedness. Yet, the relative position of the 
United States is similar between these two data sources. More extensive examination of PSTRE, 
particularly in terms of socio-demographic characteristics such as age, could inform potential 
interventions to improve technology skills of U.S. adults. Particularly, the PSTRE sub-domains 
such as technology and cognitive tasks should be examined to clarify their specific roles in the 
context of automatable jobs. Such sub-domain analysis will inform future education programs to 


improve specific PSTRE skills and to identify the lack thereof among those who work in the 
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highly automatable jobs/industry. Improving PSTRE, which can represent a transferable set of 
skills across industries, may increase the employment security in the general population and of 
middle-aged and older workers in particular. Additionally, examination of fundamental adult 
competencies (i.e., literacy and numeracy), and their relationships with the PSTRE is beneficial 
to design holistic education programs and future education policies to upgrade the sets of adult 
competencies over the life course. Finally, the routine assessment of PSTRE could be used to 
monitor the progress in the competencies and/or preparedness of the workforce in dynamic job 
markets. The OECD nations may consider incorporating the PSTRE assessment into the national 
and international surveys in future. 
Conclusion 

A skilled workforce is critical to economic growth and is especially important in a global 
economy experiencing rapid job automations. A single period of formal schooling is no longer 
adequate to keep up with changes in skills demanded by employers in the technology rich 
societies. Our analysis of PIAAC data indicated that PSTRE in the United States is significantly 
lower than most of the comparable OECD countries. It is incumbent upon us as researchers and 
educators to implement strategies and programs to ensure that adults in the United States have 
access to lifelong learning opportunities as well as continuously upgraded skills that meet the 
needs of employers, both today and in the future. 
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Figure 1. PSTRE Proficiency Levels by Country (percent) 
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Figure 2. PSTRE Proficiency Levels for Whites, Blacks, and Latinos (percent) 
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Figure 3. Use of Computer Skills at Work by Country (percent) 


Table 1. Problem Solving Proficiency Scores by Age Group 


South Korea 283* 
Germany 283* 


Singapore 287* 


Japan 294* 
Canada 282* 
Estonia 278* 
U.K. 280* 
US. 274 


Australia 289* 


G Straightforward Moderate H#Complex 


All adults 


(0.8) 
(1.0) 
(0.8) 
(1.2) 
(0.7) 
(1.0) 
(0.9) 
(1.1) 
(0.9) 


25-34 


293° 
296* 
302* 
310* 
202" 
289* 
292" 
283 

296* 


(1.6) 
(2.0) 
(1.5) 
(1.9) 
(1.5) 
(1.6) 
(1.8) 
(1.7) 
(1.6) 


35-44 


277 
25)" 
28)" 
302* 
288* 

215 
203° 

215 
291* 


(1.3) 
(1.8) 
(1.6) 
(1.7) 
(1.4) 
(1.3) 
(1.5) 
(2.0) 
(1.4) 


45-54 


261 
21a" 
211* 
202" 
274* 
200" 
212" 

266 
2037 


(1.8) 
(1.7) 
(1.8) 
(2.3) 
(1.3) 
(1.8) 
(1.8) 
(1.9) 
(1.9) 


55-65 


256 
260 
248* 
262 
261 
249* 
263 
261 
210" 


(2.8) 
(2.4) 
(2.3) 
(3.0) 
(1.4) 
(1.9) 
(2.0) 
(25) 
(1.8) 


Standard errors are shown in parentheses. 
*Significantly different from the US. 
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